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is case study examined the influences of autonomy and 
uncertain expectations of accountability on the functioning of an urban 
charter school during its initial years of development'. The school studied 
was a charter school in its second year of development that was attempting to 
measure student improvement in challenging areas such as social adjustment 
and interpersonal behaviors while working with a challenging population of 
at-risk high school students. Data were collected from a variety of sources, 
including interviews with the school's staff of four teachers, a counselor, a 
classroom assistant, and the school director. Documents generated by and 
about the school were also studied. Findings show that while requiring 
schools to administer assessments for purposes of documenting accountability 
is a worthy goal, the usefulness and appropriateness of such efforts are 
contingent on knowledgeable decision making. There were a number of problems 
in the start-up of this school, and findings show that it will be 
increasingly important to reexamine the notion of charter school autonomy so 
that the freedom to be creative and innovative does not collide with the need 
to be responsible and accountable. (SLD) 
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Issues of Accountability: A Case Study of a Charter 
School Under Development 

In recent years, politicians and educators alike have heavily debated the issues of 
educational reform, resulting in increased calls for educational accountability and 
evidence of educational effectiveness in our nation’s public schools. One arena in which 
educational accountability and evidence of effectiveness is of great concern is in the 
development of charter schools. 

Approximately 250,000 students in the United States are currently enrolled in 
charter schools (RPP International, 2000). Originally heralded as vehicles for educational 
reform and envisioned as incentives with which to improve public education, over 1,400 
charter schools currently operate throughout the nation. In essence, charter schools 
represent an experimental effort addressing the question of whether or not schools can 
succeed, if guided by a specific vision of education and if afforded a degree of autonomy 
and freedom from regulation with which to enact that vision. Part of that experimental 
effort includes holding charter schools accountable for demonstrating that they are 
achieving the outcomes of their specified vision (Dorn, 1998). 

Theoretically, charter school operators agree to allow their state’s charter school 
regulatory agency to hold them to an increased level of accountability in exchange for the 
opportunity to operate their charter schools with an increased degree of autonomy or 
freedom from regulation. However, there is some indication that for some charter schools 
the theoretical agreement (stated above), between themselves and their respective 
regulatory agencies, is not always well defined nor clearly adhered to (Griffin & 
Wohlstetter, 2001). In particular, unclear standards of accountability may be problematic 
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for some charter schools - if the performance standards to which they assume they are 
being held are not the same as those of the regulatory agency overseeing their operation 
as a charter school. In addition, preliminary findings suggest that when regulating 
agencies provide charter schools with increased autonomy (without also providing 
important forms of support and assistance), they may actually be hindering the ability of 
those schools to develop successfully and operate well (Griffin & Wohlstetter, 2001). 

These findings raise some important questions. First, is increased autonomy (e.g. 
independence and freedom from regulation) always a desirable quality for developing 
charter schools? What are the ramifications for charter schools when regulatory agencies 
are unclear about the standards to which they purport to hold charter schools 
accountable? Finally, have the necessary and sufficient conditions for successfully 
creating and sustaining charter schools yet been fully identified? 

Purpose 

In the following case study, I examine the influences of autonomy and uncertain 
expectations of accountability on the functioning of an urban charter school during its 
initial years of development. In particular, I examine from an insider’s perspective the 
effects of autonomy and uncertain accountability on the processes of creating a learning 
environment for students and a working environment for staff during the start-up years of 
an urban charter school for high-risk youth. 

Method 

The charter school selected for this case study was a desirable one for me for a 
number of reasons. First, it was a school to which I already had access and a staff that 
was willing to participate in this study. More importantly, though, it was a charter school 
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in its second year of development that was attempting to measure student improvement in 
challenging areas such as social adjustment and interpersonal behaviors and with a 
challenging population of at-risk high school students. It also was a small program (e.g. 
with an enrollment of between 40 and 60 students) that provided for me an opportunity to 
get to know well its staff and daily activities. 

This study was conducted over a period of a year and a half during which time 
data was collected from a variety of sources (Gall, Borg, & Gall, 1996; Merriam, 1998; 
and Stake, 1995). Semi-structured interviews were conducted with each member of the 
school’s staff (e.g. four certified teachers, a school counselor, a classroom assistant, and 
the school’s director); selected classes and school activities were observed; field notes 
were collected; and relevant documents generated by the charter school were examined 
(e.g. the charter school’s original charter contract; its evaluation plan; and the charter 
school’s annual progress reports, which it submits to its sponsoring agency and the state 
department of education). Member checks were also conducted with each interviewed 
staff member. 

Data was analyzed using a combination of methods described by Stake (1995) 
that included direct interpretation as well as aggregating instances of a phenomena (e.g. 
categorical aggregation); looking for corroborating instances as well as disconfirming 
instances; and searching for consistency within certain conditions (e.g. patterns and 
correspondence). 

Success for All Academy 1 



1 names have been changed 
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Success for All Academy is located in a state in the Midwest considered a pioneer 
in the charter school movement. As one of the “first generation” of states to develop 
charter schools, this state is viewed by some having “strong” charter school legislation 
(Finn, Manno, & Vanourek, 2000). However, that distinction may refer to the ease with 
which potential charter school operators in this state may obtain charter school status. 

In contrast, this state has also been described as having a fairly “weak” state-level 
accountability system (Griffin and Wohlstetter, 2001) in that it provides somewhat 
limited direction to its charter schools when those schools are in the process of setting up 
their own accountability plans. Each charter school in this state is expected to develop on 
its own an accountability plan with which to demonstrate the achievement of its goals as 
a charter school. Each operating charter school is, therefore, responsible for identifying 
its own measurable outcomes and the assessment methods to be used when documenting 
its progress in achieving the goals of its charter contract. In exchange for what some 
might perceive of as flexibility, each charter school is then held accountable by the state’s 
regulatory agency for achieving those outcomes within a three-year period of time. 

The Academy ’s Educational Program 

Now in its second year of operation as a charter school, Success for All Academy 
had for several previous years been an alternative school site for its sponsoring school 
district. Now as a charter school, it is attempting to develop itself into a two-year 
transitional program for high-risk youth in grades 9-12. Affiliated with a health and 
counseling center for at-risk adolescents, the charter school is viewed by its staff as an 
educational alternative for high school students who have not been successful in the 
traditional comprehensive high schools in the community and who may also need to 
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develop more appropriate social behaviors and/or “life skills”. The Academy may also be 
an option for students transitioning in or out of juvenile detention centers and/or chemical 
dependency programs who need an alternative educational setting while deciding whether 
or not to return to their home school. Typically, students enrolled in this charter school 
have extensive histories of school truancy (or may have dropped out of school for a 
period of time) and have significant difficulties with social adjustment and interpersonal 
behaviors. In addition, approximately 30% of its students have been previously identified 
as qualifying for special educational services for behavioral disorders and/or learning 
disabilities. 

Instructionally, this charter school offers what it refers to as an experiential 
curriculum, relying extensively on field trips and off-campus activities as learning 
opportunities for its students. It also provides a “functional” academic program, preparing 
students for the basic skills (e.g. minimal competency) tests required by the state for high 
school graduation and offering a selected range of courses through which students can 
earn credit toward high school graduation and can complete the state’s required 
performance assessment packets (an additional state graduation requirement). 

Daily Life at the Academy 

The Daily Schedule 

Students enrolled at the Academy are assigned to one of two community groups, 
depending on which staff member is their case manager. These groups meet first thing 
every morning and are used for socializing and for communicating to students the 
announcements of the day. The next two periods of the school day are academic in nature 
and include classes such as English, Math, Science, and Social Studies. The school also 
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has a “men’s group” (which all the male students enrolled in the school are to attend) and 
a “women’s group” (for all female students in the school). These groups are led by staff 
members and are apparently used for discussing gender-related issues. Although the 
Academy doesn’t have a gym, it does have a regularly scheduled physical skills class led 
by a staff member. In addition, the Academy offers classes such as Conflict Resolution, 
Health Realization, and a Latino culture class. 

Afternoons are more apt to be activity-oriented and might include guest speakers, 
class movies, art activities, and field trips. So far, students at the Academy have taken 
field trips to places such as an Institute of Art, the state’s Historical Center, a water 
treatment plant, and an indoor rock climbing facility. In addition, some students leave for 
work-study responsibilities in the afternoons. 

Student Absenteeism 

Staff members indicate that they never know how many students will show up for 
their classes until the class starts. Some times all the chairs are full. Other times, only a 
few are in use. I would learn later on that having a history of poor school attendance is 
one of the “entrance’ criteria for students enrolling in this school. 

Teachers ' Roles and Responsibilities 

Like the other teachers at the Academy, Suzanne 1 is expected to teach a wide 
range of subjects. In addition to teaching English (which is her major), she was also 
teaching a Health class, a class called “Journey through Latino Culture”, and a class 
called “Health Realization” which Suzanne described as “more of a philosophy class on 
how your thoughts can make your reality.” 



1 names have been changed 
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As a staff member of the academy, Suzanne is also expected to carry out multiple 

roles. 

“Generally speaking, I’m a teacher. And I generally teach three to four 
different classes a semester. One of those classes is actually in my field. 

So, I am required to teach outside of my field. In addition to that, we do 
case management where I have anywhere between ten to fifteen students 
and just kind of help manage their lives. And then we also do quite a bit of 
team collaboration as far as program development and curriculum 
development. And then I do some administrative stuff as well - paperwork 
stuff, tracking... and then also establishing relationships with parents. So 
there’s a lot going on.” 

Students in the Program 

Suzanne invited me to observe her English class. I entered the room where her 
class was to take place and sat in a chair in what I thought was the back of the room. 
Students soon began to shuffle into the room and then collapse into the chairs at the 
tables. Students didn’t appear to have assigned seats. Some students sat together. Other 
students seemed to seek out places to be alone. For the most part, they seemed kind of 
subdued - not much talking. Initially, thirteen students entered the room (e.g. it was about 
9: 15 A.M. or a few minutes after) - five females and eight males. Later in the period, an 
additional male and female would arrive. I think each of the late arrivers brought “late 
passes” with them as they entered the room. 

I noticed that several of the students kept their jackets and/or parkas on while in 
the classroom even though there were lockers lining the hallway outside of the room. 
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Since Suzanne didn’t say anything to the students about taking their jackets off, I’m 
assuming that the school must allow “jackets on” in the classroom. One of the female 
students brought a purse with her into the room. In addition, several students had cans of 
pop and/or bags of chips with them as they entered the classroom and ate their chips 
throughout the period. One female student had something in a paper bag - like a bagel - 
that she took bites from during part of the period. 

I found myself noticing these things, I think, because the school at which I used to 
teach (a special setting for severely E/BD secondary students) had rules that didn’t allow 
jackets, hats, purses, food, etc. in the classroom. So, to see those previously “forbidden” 
things in this classroom caught my eye. As expected, the physical appearance of some 
students was eye-catching. Some students had nose rings, one male student had tattoos 
visible on his arm, and one female student had her hair dyed pink. 

As I sat in my chair, I could overhear some students talking among themselves at 
their tables. I noticed one male student displaying his middle finger to another student 
(when Suzanne wasn’t watching). The finger gesture must have been done in jest as the 
other student seemed to laugh it off. Occasionally, I could hear students using terms such 
as “f — ” - especially at a table at the opposite end of the room from where I was seated. 
When Suzanne heard language such as that, she would say something like “Watch that 
language” or “My ears are burning” in kind of a joking way. Ironically, students seemed 
to cease using those terms when she made a comment about it. In general, most students 
seemed tired or very lethargic. 

Academic Instruction at the Academy 
A Well-Prepared Instructor 
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Before students entered the classroom, Suzanne had written the following 
questions on one of the blackboards. 1) Why do young people use/abuse drugs? 2) What 
can adults do about it? Earlier in the week, Suzanne’s students had written a draft of an 
essay that was to be a response to one of the questions on the board. As an added 
incentive for her students, Suzanne had informed her students that she was planning to 
enter the final versions of their essays in the local newspaper’s writing contest. Therefore, 
during today’s class, Suzanne was going to have the students work independently on a 
series of worksheets while she held individual conferences out in the hallway and 
conferred with each student about their respective essays - giving them feedback on their 
first drafts and preparing them for their next draft. 

Suzanne had also prepared the following materials ahead of time and had laid 
them out on the table near where I was sitting: 

1 . A pile of folders and notebooks. “For students who would need them,” Suzanne 
indicated. I assumed she meant those students who maybe didn’t bring their own 
notebooks to class. 

2. Graded assignments (possibly from a previous day). 

3. Scrap paper. Suzanne indicated that students were going to write a “warm-up” 
writing assignment later in the period. 

4. A four-page set of handouts. These were the series of worksheets that students 
were going to work on independently while Suzanne conducts individual 
conferences with students in the hall. 

Suzanne 's English Lesson 
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Students were now seated in the chairs around the four tables. Suzanne walked to 
the “front” of the room - near where I was seated. She began her class by introducing me 
to the students as a visitor in their class that morning. The students did not seem to react 
to that announcement at all. In fact, they didn’t even seem to look at me. I wonder if this 
is an indicator of indifference on their part? Or is their lack of reaction just on the 
surface? Is it just not cool to react to a visitor? 

Suzanne handed out notebooks and folders to students who hadn’t brought 
anything to class - these are apparently the students “who needed them”. The graded 
papers were also handed back to students. Next, Suzanne reviewed with students her 
plans for that period and for the next day. Students would be working independently 
(Suzanne stressed that several times) on the four handouts stapled together while she 
conferenced with each of them individually in the hallway. She stressed the self- 
directedness of the assignment and her expectation that students would work in a self- 
directed way on their own while she conferenced in the hall. Suzanne also reviewed with 
students the newspaper assignment that would be the focus of their conferencing. A few 
of the students asked her if they had actually written a draft. Yes, they had, Suzanne 
responded. Those students still seemed confused - as if they couldn’t remember doing the 
first draft. 

Suzanne then assigned students the warm-up writing assignment. She asked them, 
“Have you ever been judged wrongly by an adult because of your age - such as when at a 
mall or like when on the museum trip yesterday?” (Apparently, the school had taken a 
field trip to a museum the previous day). “Or if you can’t think of a specific incident,” 
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continued Suzanne, “then respond in general to the question: Do adults judge young 
people? If so, why?” 

Some students began to mildly tease Suzanne at this point since she had repeated 
the directions to the warm-up writing assignment multiple times by now. It seemed to be 
good-natured joking and Suzanne seemed to accept that joking as O.K. Suzanne then 
called out into the hallway the first student with whom she was going to begin 
conferencing. The other students gradually (and on their own, for the most part) began 
writing a response to the warm-up question. As they finished their responses, they handed 
them to Suzanne (as she came and went from the room between subsequent conferences 
with students in the hall). 

Students gradually began working on their four-page assignment. About this time, 
I heard Suzanne say to a student (on one of her trips back into the classroom): “You need 
to finish this (the warm-up assignment) by using periods rather than ‘and - and - and’ all 
the way through (the written response)”. I noticed that Suzanne then gave the warm-up 
writing assignment back to that student (presumably to redo). 

The four-page assignment dealt with drug use. Each page contained several 
paragraphs of text (e.g. four or five) that students were to read. Each page also included 
one or more photographs illustrating the content of the text. One page contained 
information on a graph along with three or four paragraphs of text. Another page had 
some paragraphs of text along with a few short-answer questions that students were to 
answer with a written response. The last page contained some paragraphs of text and an 
essay-type question. Suzanne (when preparing these materials ahead of time) had taken a 
black marker and written explicit instructions and prompts on each page indicating what 
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parts of the pages were to be filled out by the students (e g.” DO” - written in black 
marker on some pages, apparently indicating “do these questions”). On the last page - 
instructing students to answer the essay question by writing their response on the back of 
the page - was the black marker directive: “Now answer on back”. 

Suzanne’s English class ended at 10:15 A.M. and students were directed to hand 
in their folders and clean up their tables. Students followed the directions in kind of a 
sluggish way and then shuffled out of the room - apparently for their break. I was to find 
out later that students were allowed to go outside between class periods and smoke. 
Instructional Goals 

Suzanne mentioned later that her English lesson had two purposes. First, to get 
her students to complete a task and, secondly, to get them to think about a topic of great 
relevance to them personally. 

. .to feel comfortable talking about a really hot topic for these kids. I 
mean, it is drug use. So that they can actually feel comfortable enough 
talking about drug use and then writing an essay about it as well. Because, 
some of them, it’s a very personal issue.” 

I was surprised, since she was teaching an English class, that none of her goals focused 
on written composition. However I was to learn later on that the Academy encourages its 
staff to develop assignments around issues relevant to students such as drug use. 
Academic and Behavioral Expectations 

As I reflected back on Suzanne’s class, I noticed that the students behaved fairly 
well while in the classroom with her. Students, for example, seemed familiar with 
Suzanne’s classroom routines and expectations and seemed to comply with her requests 
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(at least in today’s class). Students followed her directions fairly well and ceased using 
inappropriate language when asked. They began their work when prompted and handed 
in their folders at the end of the period when directed and without getting out of control 
(in today’s class, at least). Students also came into the room and sat down without being 
told. I am assuming that Suzanne has had to teach and enforce those expectations 
consistently since the beginning of the school year. 

Academic Curriculum at the Academy 

Suzanne indicated that there is no one textbook or teacher’s manual that she uses 
when planning her lessons or when planning for her courses. Instead, she is apparently 
given a lot of freedom to use her own judgment and to be, as she would say, creative. 
This also appears to be the case when she is teaching courses outside of her major. 
“There are books for ideas. I don’t have one book that I go by. Nothing is 
mapped out for me. It’s more like I come up with the objectives for this 
course. And then how can I reach those? So then, I can draw from 
different sources. . .there is no one book. We have a lot of different stuff in 
hard cover. There’s a ton of stuff on the Internet. . but it’s very creative as 
far as how the lesson plans are written.” 

Suzanne explained that she gets ideas for the objectives of her courses from the state 
graduation standards, the state basic skills tests, and from what she perceives to be the 
needs of her students. 

“. . . kind of meeting them where their needs are at. So if they’re not 
passing the writing tests, I need to make sure that my English class covers 
how to write an essay. “ 
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Suzanne seems pleased with the freedom that she has to be creative with the curriculum. I 
wonder, though, if each teacher at the Academy has as much curricular freedom as 
Suzanne, then how does one go about articulating the school’s curriculum? And if the 
curriculum cannot be articulated, them how does one go about assessing outcomes from 
it? 

Course Planning at the Academy 

Although Suzanne knows what she will be teaching during the upcoming term 
(English, Health, and Health Realization), she does not yet know what she will be 
teaching later in the school year. “We’re still planning that, Suzanne explained. “For 
sure, English.” Apparently, the staff makes their decision as to what courses to offer 
based on a number of factors such as the requirements of the state graduation standards, 
the background of their current teaching staff, and the types of courses for which their 
students need credits. According to Suzanne, the decisions will be made: 

“. . .based on the graduation standards plus the team plus what the 
students’ need. We haven’t offered American History for a while so that 
might be something that we’d have to offer. . . Students need to have so 
many credits in English. So we have to make sure that we provide that. 

And then like the American History, there’s supposed to be two credits in 
that, so if we look at our students and they’re low on the American History 
and we haven’t offered it for a while, that might be something that we’d 
have to offer.” 

After listening to Suzanne, I get the impression that the staff at the Academy has been 
given a lot of freedom with which to plan its curriculum. I am also getting the impression 
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that this school is still in its infancy in terms of articulating its curriculum. I wonder why 
the Academy seems to still be searching for a curriculum since by now they are starting 
their third year of existence as a charter school. 

Findings 

Charter School Autonomy and Assessment Decision Making 
Selecting Realistic and Measurable Student Outcomes 

As mentioned earlier, Success for All Academy is located in a state where charter 
schools are expected on their own to select outcomes for which they will be accountable. 
To some, this freedom may be viewed as an asset. However, for others it may be a 
stumbling block. 

Charter contract. Success for All Academy’s charter contract sets forth an 
ambitious agenda. It stipulates, first of all, that 80% of the Academy’s students will 
demonstrate improvement in: 1) school attendance, 2) accumulation of credits toward 
graduation, and 3) academic gains on standardized tests. Given that the Academy is a 
high school, the above expectations appear to be very appropriate goals for which to be 
held accountable. Although the expectations above could be potentially challenging ones 
in which to demonstrate student improvement (considering the histories of the students 
enrolled in this charter school) they still appear to be outcomes on which measurable data 
could be collected in a fairly reasonable manner. 

However, the fourth area of the Academy’s contract presents a far more 
complicated situation. It stipulates that 80% of students will demonstrate improvement in 
the following twelve “broad outcome goals”: 1) gaining effective communication skills, 
2) gaining life skills and social skills, 3) developing a sense of interconnectedness of self 
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and world community, 4) developing goal setting, achievement and employment skills, 5) 
valuing and contributing to a multicultural society, 6) becoming a productive/responsible 
citizen, 7) gaining science and mathematics skills, 8) developing the ability to access 
information, 9) acquiring critical thinking skills, 10) demonstrating self-expression 
through healthy physical, creative and leisure activities, 11) gaining increased self 
awareness, self esteem and self respect, and 12) gaining increased resiliency attributes. 

Ambitious goals. While the above twelve broad outcome goals certainly represent 
important areas of growth for students, they also seem to present fundamental problems 
for the Academy. First, the outcome goals appear to be an extremely broad and lengthy 
set of expectations. It’s questionable if the twelve goals could be realistically 
accomplished in any high school, much less in a two-year transition program. In addition, 
trying to operationalize and collect measurable data on each of the goals would appear to 
be a daunting - if not impossible - task also. Finally, if Success for All Academy plans to 
assess its students on each of the above broad outcome goals, then one would assume that 
it also plans to incorporate into its curriculum opportunities for students to learn the skills 
and capacities listed in each goal. That, in and of itself, would appear to be a monumental 
task for a small developing charter school. 

Ironically, Success for All Academy is not alone in setting ambitious goals for 
itself as a charter school. Griffin and Wohlstetter (2001) reported that many of the charter 
schools in their study (also from “first generation” charter school states) included in their 
charter applications outcomes that were also difficult to define and measure. 
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Selecting a Measure for Assessing Student Outcomes 

Selecting an assessment instrument and an appropriate method of measurement is 
again something that charter schools in this state are expected to do on their own. 
Therefore, having some basic understanding of assessment issues might help some 
charter schools ask key questions about potential strengths and limitations of 
measurement products - such as how generalizable the instrument is to other settings and 
how readily the instrument can be implemented into a school setting. In addition, having 
some basic knowledge of appropriate assessment practices may also help some charter 
schools use instruments in intended ways and in ways that will enhance (rather than 
detract from) the degree of validity and reliability of the data collected. In a sense, 
becoming a wise consumer of assessment products may be of value for some charter 
schools, such as Success for All Academy. 

Faced with the decision of identifying a measure with which to monitor student 
progress on the broad outcome goals of their charter contract, the leadership of Success 
for All Academy decided to adopt the AUEN (Assessing Unique Educational Needs), a 
standards-driven assessment system developed by the American Institutes for Research 
and Disability Research Systems, Inc. (1996). The leadership of the Academy felt that 
the performance standards of the AUEN (which focused on major life goals of adulthood) 
best reflected its own broad outcome goals and would therefore be an appropriate 
instrument for collecting performance data on students. 

Strengths and limitations of a measure. In contrast to the somewhat loosely 
defined goals in the Academy’s contract, the AUEN had clearly stated performance 
standards that addressed nine major life goals of adulthood such as interpersonal 




19 



Issues of Accountability 



19 



effectiveness, work habits, and selected cognitive processes. In addition, the AUEN 
included a system for rating students’ performance according to specific performance 
requirements for each standard. 

The AUEN was originally developed as an alternative assessment system for 
students in special education settings. Several districts in selected states throughout the 
U.S. currently use various versions of the AUEN for that purpose. However, the Success 
for All Academy was the first school to decide to use the AUEN with at-risk students and 
the first charter school to attempt to use it as a measure of school accountability. While 
some research existed on the use of the AUEN as an alternative assessment with special 
education populations, no research had yet been conducted on the use of the AUEN with 
at-risk students or on the use of the AUEN as a measure of charter school accountability. 

Feasibility. The version of the AUEN adopted by the Academy included 
additional methods for collecting observational data on student performance. To a certain 
extent, this version of the AUEN was a type of “proof-of-concepf ’ prototype in that the 
developers of the AUEN were attempting to develop methods with which school 
personnel could collect observational data that would have high levels of technical 
adequacy and which theoretically could be used to enhance the quality of the data from 
the rating system. Although several years of effort had gone into identifying appropriate 
performance standards for the AUEN and time had been devoted to designing these 
additional data collection methods, little information had been gathered yet on the 
feasibility of implementing these more complex methods into actual school settings. That 
is, no information existed on the practicality of having school personnel learn to use the 
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suggested observational procedures included in this version of the AUEN and to 
incorporate those procedures into the daily activities of an educational program. 

Missing component. While all versions of the AUEN included performance 
standards (and the version adopted by the Academy also included suggested methods for 
collecting observational data), none of the AUEN versions included a curriculum for 
teaching to students the interpersonal skills, work habits, and cognitive processes 
addressed in the standards. Instead, the developers of the AUEN assumed that those 
schools adopting their materials would already have in place a successfully operating 
curriculum through which the performance standards could be achieved. 

Asking the right questions. Although the standards of the AUEN were well 
defined and seemed to be congruent with several of the Academy’s broad outcome goals, 
there were other features of this measurement system that the Academy could have 
viewed with caution. The version of the AUEN adopted by the Academy, for example, 
included untried and complex methods for collecting observational data. Not only was 
the feasibility of teaching school personnel to learn to use and implement those methods 
yet unknown but the question of whether those more complex procedures would actually 
enhance the quality of student ratings was also yet to be answered. More importantly, the 
standards and data collection methods of the AUEN were intended for use in schools with 
established curricula already in place and functioning well. Yet the Academy’s staff was 
still struggling to define their courses and to create content for the courses they currently 
offered. 

It is unknown how well the limitations of the AUEN were explained to and 
understood by the leadership of the Academy. In retrospect, it also seems somewhat 
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unsettling that an assessment system with unproven procedures would be suggested for 
use at all. However, it’s possible that the Academy leadership, when left on their own to 
make a decision, were unaware of the ramifications of these limitations or were too 
overwhelmed with other responsibilities and the perceived pressure to produce 
accountability data that they overlooked the AUEN’s potential shortcomings and 
proceeded ahead anyway. 

As it would turn out, the observational methods of this version of the AUEN were 
indeed overwhelming for the staff of the Academy to learn to use and to implement - at 
least in the manner intended by its developers. In addition, without any consistent 
curriculum in place at the Academy during this time, it was difficult to know to what to 
attribute any improvement in student performance - once data had been collected. So, 
although the Academy’s staff went through the motions of using some of the AUEN 
materials to gather data on their students and included that data in their required annual 
reports and sponsorship review, the value of the data was definitely questionable. 

Autonomy and Charter School Management 

Although the Academy had been a charter school for two years, those early years 
had not been without problems. The freedom of the charter school to hire its own staff 
members and to manage its daily responsibilities as a school became almost unbearable at 
times. Possibly this is another area where external support would have been helpful - 
especially for a small school such as Success for All Academy. It appears that the 
Academy could have benefited from support in making management decisions that would 
have created a more stable working environment for its staff members and support in 
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making curricular decisions that would have resulted in a more stable learning 
environment for its students. 

Personnel Issues 

According to staff, this charter school lacked stable leadership during its 
beginning years. The Academy had two different directors during its first two years and 
was without a permanent director at the time the AUEN was adopted. During that time of 
changing leadership, the charter school was also chronically understaffed - with staff 
positions unfilled or filled multiple times throughout each year as a constant stream of 
staff members were hired and then quit. Staff reported that they had three different social 
workers come and go during one semester alone. The unfilled positions left the teachers 
who remained without adequate support in the classroom. The constant change in 
personnel made it virtually impossible to develop any continuity in the school’s 
educational program. 

Job Stress and Confusion 

In general, staff reported that those early years were characterized by stress, 
instability, disorganization, and serious student behavior problems. In addition, staff 
indicated that their workloads during those first two years were unbearable. As one 
teacher described it, “Last year I think I was teaching nine courses per semester, “ she 
explained. . . That was crazy.” Not only were they assigned too many classes to teach, 
they weren’t given enough time to adequately plan and prepare for their assigned classes. 
The Academy was also operating on a year-round basis during that time. In order to get 
in vacation time, staff had to take days off when school was in session - leaving the 
school short-staffed during their absence. 
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Staff indicated that the Academy’s curriculum underwent repeated changes during 
its beginning years. According to the staff, these continual cycles of change were also 
extremely stressful and resulted in a continuous flow of new expectations that they were 
expected to carry out. 

“[It] seemed more like day-to-day survival . . . like we were just trying to 
get through the day. . . just keeping things together day by day by day. . . 
and . . . there wasn’t a lot of planning for the future.. . . We’d been 
understaffed and we needed another teacher. The teachers are teaching 
outside of their subjects in multiple courses....” 

It was not surprising that they reported feeling totally overwhelmed when 
introduced to the AUEN, which was for them yet one more new program to try to 
understand and assimilate. As one teacher described it: 

“It was kind of aggravating because I don’t think the school was prepared 
to take on another really complex goal. It seemed there was too much 
going on at once. And this was a new system and it was very complex. 

And in order for it to work it has to be done properly. So, at first, it was 
kind of just aggravating because I don’t think we were ready.” 

At the same time that the Academy was introducing its staff to the AUEN, it was 
also trying to implement a new computer-based instructional program. On top of 
everything else, the Academy had mistakenly enrolled several students with seriously 
aggressive behaviors during that time - the types of serious behaviors that they as a 
school were not prepared to deal with. One student, for example, brought a weapon to 
school and, in another incident, a fairly serious fight broke out between one of the 
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Academy’s students and someone from outside the school. Those events were quite 
traumatizing for both students and staff. According to one staff member: 

“[There were] too many things up in the air. And it was a transition with 
the new director. And there were new students that were kind of shuffled 
in here quite quickly. And there were a lot of repercussions - behavioral 
problems - that happened during that period.” 

In retrospect, it appears that the Academy would have benefited from persons in 
leadership positions who were more knowledgeable about managing the daily operations 
of a school and had a more realistic understanding of the nature of teaching and the type 
of support that teachers need in order to do well in the classroom with their students. 
Hearing of the early experiences of the Academy also raises questions about the notion of 
accountability. Should the Academy’s staff have had to live through the extreme 
experiences that they did during those early years? Should someone have been 
accountable for monitoring the Academy’s working and learning conditions during that 
time? Is accountability relevant only in discussions of student outcomes? 

Uncertain Expectations of Accountability 
Preparing for Contract Review 

According to the Academy’s charter contract, one of the grounds for terminating 
its contract is “failure to meet requirements for pupil performance.” Therefore, part of 
the Academy’s motivation in deciding to adopt a measure (e g. the AUEN), was its 
assumption that it needed to be reporting to the state and to their sponsoring agency 
information on their students’ progress. Since the Academy had heard nothing from the 
state or from their sponsoring agency, they felt obligated to continue to adhere to the 
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expectations of their contract and assume that, in order to be eligible for the renewal of 
their contract, they needed to have student performance data to show to their sponsoring 
agency. The fact that their curriculum was not yet developed (thereby making any 
collected performance data difficult to interpret) seemed to not be an important matter at 
this point. It was as if they were willing to collect data so that they would have 
“something” to show to their sponsor and the state’s regulatory agency even if that data 
was of questionable value. Understandably, doing whatever it took to maintain their 
charter contract was important to them even if it meant doing something that didn’t make 
sense in the long run. 

As it turned out, neither the state nor their sponsoring agency held them 
accountable for providing data in those areas of the curriculum that were still under 
development. Unfortunately, though, neither agency communicated that change in 
expectation to the Academy ahead of time. Therefore, without a clear understanding of 
the expectations that others had of them, Academy ended up devoting a lot of 
unnecessary staff time and effort to collecting unnecessary data that was of questionable 
value. Given all the other tasks facing them during that period of time, their time could 
have been much better spent on activities such as curriculum development than on 
collecting questionable data. 

Adhering to Expectations Whatever the Cost 

During the time that the staff of the Academy was intently trying to use the 
AUEN materials to collect student data (in an attempt to adhere to the expectations they 
assumed others had of them), they were also deluged with a series of other dilemmas 
such as attempting to establish a stable work force, developing a more realistic work load 
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for each staff member, and creating a more well-defined curriculum for students. In 
attempting to accomplish the task of collecting data while also addressing the needs of 
their overworked teaching staff, the Academy simplified the assessment procedures of the 
AUEN to make it better fit their time constraints. 

Specifically, they decided to assign the assessment responsibilities to their support 
staff who had less hectic daily schedules instead of having their teachers complete the 
AUEN ratings. Unfortunately, the support staff also had the least amount of training in 
the use of the newly introduced assessment system and tended to have the least amount of 
consistent contact with students. While staff seemed aware of the limitations of assigning 
assessment responsibilities to their support personnel, they also saw it as a necessary 
solution to their dilemma of over-worked staff members. As one of the teachers explained 
it: 

“... the teachers don’t have any time just within the daily schedule. Our 
prep time is very minimum. And in order to have good days with good 
behavior the lesson has to be very good and very structured. And that 
takes time. And that takes a clear head. So, the very little prep time that we 
had - for us to do the ratings as well - our classes would fall apart.” 

While the above decision created methodological problems for them and called into 
question again the adequacy of their data, it did help them deal with their immediate 
needs. In addition, staff members seemed to view that decision as a logical and welcome 
solution. 

The irony of this situation is that the collected data, regardless of its quality, 
wasn’t necessary after all given the change in expectations of the state and the 
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Academy’s sponsoring agency. However, without that knowledge ahead of time, the staff 
devoted considerable time and effort into finding a solution to a problem that ultimately 
didn’t exist. 

Conclusion 

In theory, requiring charter schools to administer assessments for purposes of 
documenting accountability is a worthy goal. However, the usefulness and 
appropriateness of such efforts are contingent upon knowledgeable decision-making. 
Operators of charter schools need adequate knowledge with which to make decisions 
about their assessment activities. Policy-makers and charter school regulators need an 
adequate understanding of the conditions under which charter schools will function most 
successfully. As calls for educational accountability increase, so too will the need for 
professionally responsible ways of implementing assessment activities and providing 
critical forms of support to developing charter schools. In addition, it will be increasingly 
important to re-examine carefully the notion of charter school autonomy so that the 
freedom to be creative and innovative does not collide with the need to be responsible 
and accountable. 
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